Storage and Processing Systems for Power - Law Graphs

Large graphs abound around us – online social networks, Web graphs, the Internet, citation networks, protein interaction networks, telephone call graphs, peer-to-peer overlay networks, electric power grid networks, etc. Many reallife graphs are power-law graphs. A fundamental challenge in today’s Big Data world is storage and processing of these large-scale power-law graphs. In this thesis, we show that graph processing can be made faster and graph storage can be made more efficient by using techniques that leverage the structure of the underlying power-law graphs. To this end, we present two systems. First, we present LFGraph, which is a fast, distributed, inmemory graph analytics platform. LFGraph leverages the structure and characteristics of power-law graphs in order to reduce communication overhead, and to balance communication and computation load. This makes analytics faster on power-law graphs. Next, we present Bondhu, which is a disk layout manager for graph databases. Bondhu exploits the fact that most real-life power-law graphs are also small-world and these exhibit strong community structure. Bondhu utilizes this community structure in order to make layout decisions. This improves the query response time of graph databases. Our systems are evaluated on real clusters using real-world graphs.